An Analysis of Non-Linearities in Neural ICA Algorithms

The non-linearities in the objective functions play an important role in the convergence and stability of various neural ICA algorithms. In case of maximization of nongaussianity, they influence the negentropy and in Maximum Likelihood Estimation (MLE), they are related to the assumed distributions of sources. We present in this paper an experimental investigation of the performance of such non-linearities using artificially generated signals with known statistical properties. From a comparison between modified fixed-point iteration and natural gradient ICA neural algorithms, it was found that the fixedpoint iteration algorithm has a significantly better performance and was thus selected as a platform for the comparison of the various non-linearities. To the commonly used tanh and Gaussian non-linearities, we have introduced two others; one is based on the Cauchy distribution and the other is based on the Exponential Power Distribution (EPD). A large number of experiments have been done with artificially generated sources under the conditions of superand sub-Gaussian source signals using the various non-linearities. From the measurement of the asymptotic variance of the unmixing matrix and the convergence profiles, it is found that the Cauchy non-linearity performs better than all the others in the case of small numbers (<12) of super-gaussian sources. With large numbers of sources, the performance of such non-linearity is found to be poorest while the tanh and EPD nonlinearities exhibit the best performance for both superand sub-Gaussian sources. However, the EPD non-linearity has the advantage of parameterization and so it can be used more flexibly with sub-gaussian signals.